Display control apparatus and imaging apparatus

ABSTRACT

A display control apparatus includes a focus detector configured to detect a focus state based on an image signal acquired from an imaging part, a main object detector configured to detect a main object among objects in an image based on the image signal output from the imaging part, and a display controller configured to display on a display unit an index representing the focus state detected by the focus detector on the main object detected by the main object detector in manual focusing. The display controller controls switching of the main object in the manual focusing.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to a display control apparatus and animaging apparatus configured to detect a focus state and to control adisplay of the focus state.

Description of the Related Art

A focusing apparatus in a recent high-resolution video camera etc.compatible with the full high vision and 4K has a resolution higher thanthat of prior art, and it is not easy for a photographer to strictlyfocus on an object through manual focusing operation (“MF” operation).In particular, when the photographer performs focusing while confirmingthe object through a viewfinder, a display panel, etc., a defocus mayoccur which cannot be recognized by the viewfinder, the display panel,etc., and it is difficult to determine whether the intended focus stateis obtained.

A focus assist method for assisting the MF operation has recently beenproposed. Japanese Patent Laid-Open No. (“JP”) 2007-248615 discloses amethod for calculating an in-focus evaluation value in an MF operationand for displaying an in-focus degree through a bar. JP 2005-140943discloses, as a focus assisting method for an imaging apparatus, aplurality of display methods that represent a change of a focus state asa focus lens moves. JP 2001-083407 discloses, as a focus state detectingmethod, an imaging apparatus that provides an imaging plane phasedifference detection method based on a live-view mode used to capture animage while the image is displayed on a back motor, etc.

Another proposed camera serves to determine a target face (main face)based on a position and a size of the face automatically recognized witha face detection function where there are a plurality of human objectsin capturing a person. This camera sets a focus detecting area for amain face and performs a focus detecting process.

A camera is disadvantageous that serves to automatically determine amain face among a plurality of faces with a face detecting function andprovides a focus assisting function and a display control to the mainface selected among the plurality of human objects.

A main human object to be captured by the photographer is often disposedat the center of an image. Since the main human object is likely closerto the camera than other objects, the main human object is likelylargest among objects. When the camera determines as a main face a largeobject located at or near the center of the image in accordance with thephotographer's intent, the photographer does not feel strange.

However, for example, in capturing a motion image in a crowded scene,another larger face may approach to the center position than the currentmain face in the current image of the camera. When the cameraautomatically switches the main face to the other face while thephotographer provides the MF operation on the current main face throughthe focus assisting function, the target for the focus assisting displaymay become the other face. As a result, manual focusing targeted by thephotographer on the object fails.

In addition, an imaging scene with many persons contains many faces inthe image and causes the main face to be frequently switched. Moreover,as another face crosses in front of the main face, the main face may beswitched. When the main face is switched while the focus assistingdisplay and the MF operation on the main face are used, the photographermay feel strange or unpleasant.

SUMMARY OF THE INVENTION

The present invention provides a display control apparatus and animaging apparatus for a stable focus assisting function in manualfocusing which can relieve a photographer from feeling strange orunpleasant.

A display control apparatus according to the present invention includesa focus detector configured to detect a focus state based on an imagesignal acquired from an imaging part, a main object detector configuredto detect a main object among objects in an image based on the imagesignal output from the imaging part, and a display controller configuredto display on a display unit an index representing the focus statedetected by the focus detector on the main object detected by the mainobject detector in manual focusing. The display controller controlsswitching of the main object in the manual focusing.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a schematic structure in an imaging systemaccording to embodiments of the present invention.

FIG. 2 is a schematic diagram of a pixel array in an image sensor.

FIGS. 3A-3D illustrate examples of a focus assisting display accordingto a first embodiment.

FIG. 4 is a flowchart of a focus assisting display control methodaccording to the first embodiment.

FIG. 5 is a flowchart of a focus detecting process according to thefirst embodiment.

FIGS. 6A-6D illustrate examples of a focus detecting area and an imagesignal obtained from the focus detecting area.

FIGS. 7A and 7B explain a correlation calculating method according tothe first embodiment.

FIGS. 8A and 8B explain a correlation calculating method according tothe first embodiment.

FIG. 9 illustrates a relationship between a defocus amount and a displaypart position for the focus assisting display according to the firstembodiment.

FIG. 10 is a flowchart of a face information acquiring process accordingto the first embodiment.

FIG. 11 is a flowchart of a main face determining process according tothe first embodiment.

FIG. 12 is a flowchart of a face correlation determining processaccording to the first embodiment.

FIG. 13 is a flowchart of the face information acquiring processaccording to a variation of the first embodiment.

FIG. 14 is a flowchart of the face information acquiring processaccording to a variation of a second embodiment.

DESCRIPTION OF THE EMBODIMENTS

(Structure of Imaging System)

FIG. 1 is a block diagram of a schematic structure in an imaging system1 having a focus assisting function according to embodiments of thepresent invention. The imaging system 1 includes a lens unit 10 and acamera body 20. While this embodiment detachably attaches the lens unit10 to the camera body, but the lens unit 10 may be integrated with thecamera body 20.

A description will now be given of the structure of the lens unit 10.The lens unit 10 includes a fixed lens 101, a diaphragm (aperture stop)102, a focus (or focusing) lens 103, an unillustrated zoom (or zooming)lens, etc. The diaphragm 102 is driven by a diaphragm controller 104,and controls an incident light quantity on an image sensor 201, whichwill be described later. The focus lens 103 is driven by a focus lensdriver 105 for focusing. The zoom lens is driven by an unillustratedzoom lens driver for zooming. While this embodiment 10 includes the zoomlens and the zoom lens driver, these components are not indispensable tothe present invention and may be omitted.

A lens controller 106 integrally controls operations of the entire lensunit 10, and communicates data with a camera controller 207 thatintegrally controls entire operations of the entire imaging system 1.The lens controller 106 controls the diaphragm driver 104, the focuslens driver 105, and the zoom lens driver in accordance with controlcommands and control information received form the camera controller207, and sends lens information to the camera controller 207. The lenscontroller 106 controls the diaphragm driver 104, the focus lens driver105, and the zoom lens driver, and thereby an aperture diameter in thediaphragm 102, the positions of the focus lens 103 and the zoom lens. Inaddition, the lens controller 106 provides a control in accordance witha user operation for focusing, zooming etc., when the user manipulates afocus ring, a zoom ring etc. in a lens operating unit 107.

A description will now be given of the structure in the camera body 20having the focus assisting function. The image sensor 201 includes a CCDor CMOS sensor, and the light flux from the imaging optical system inthe lens unit 10 forms an image on a light receiving plane on the imagesensor 201. The formed object image is photoelectrically converted intoelectric charges in accordance with the incident light amount and storedby photodiodes (photoelectric converters) in the image sensor 201. Theelectric charges stored by each photodiode is sequentially read as avoltage signal corresponding to the electric charges out of the imagesensor 201 based on a driving pulse given by a timing generator 210 inaccordance with a command from the camera controller 207. The detailedstructure of the image sensor 201 will be described later, but the imagesensor 201 according to this embodiment can output a pair of focusingsignals usable for a phase difference type focus detection as well as ausual capture signal.

The capture signal and focusing signal read out of the image sensor 201are input into a CDS/AGC circuit 202. The CDS/AGC circuit 202 performscorrelated double sampling for removing reset noises, a gain control,and a signal digitalization. The CDS/AGC circuit 202 outputs a processedcapture signal to a camera signal processor 203, and the processedfocusing signal to a focus signal processor 204.

The camera signal processor 203 performs various image processing forthe capture signal output from the CDS/AGC circuit 202, and generates animage signal. The display unit 205 is a display device, such as an LCDand an organic EL, and displays an image based on the image signaloutput from the camera signal processor 203. When the camera body 20 isset to a recording mode for recording the capture signal, the capturesignal is sent from the camera signal processor 203 to a recorder 206and recorded in a recording medium, such as an optical disc, asemiconductor memory, a magnetic tape.

The focus signal processor 204 detects a focus state through acorrelation calculation based on a pair of focusing signals output fromthe CDS/AGC circuit 202. In this embodiment, the focus signal processor204 calculates the focus state, such as a correlation amount, a defocusamount, and reliability information (coincidence between two images(“two-image coincidence”), steepness of two images (“two-imagesteepness”), contrast information, saturation information, and scratchinformation). The focus signal processor 204 outputs the calculateddefocus amount and reliability information to the camera controller 207.The camera controller 207 notifies the focus signal processor 204 of asetting change for calculating the defocus amount and reliability basedon the acquired defocus amount the reliability information.

The camera controller 207 communicates information with and providescontrols over each component in the camera body 20. In addition, thecamera controller 207 controls the power on/off, the setting change, andrecording in accordance with the input from the camera operating unit208 operated by the user as well as controlling processes in the camerabody 20. Moreover, the camera controller 207 performs a variety offunctions in accordance with the user operation, such as switchingbetween autofocus (AF) control and manual focus (MF) control and aconfirmation of a recorded image. In addition, the camera controller 207communicates information with the lens controller 106 so as to sendinformation of a control command and control information for the imagingoptical system, and to acquire information in the lens unit 10.

A face detection processing circuit or processor (main object detector)209 performs a face recognition process for an image signal output fromthe camera signal processor 203 and detects, as an object area, a facearea of a main object in a captured image (or performs a face detectingprocess). One conventional face recognition process is, for example, amethod for detecting a skin color area from a gradation color in eachpixel expressed by image data corresponding to an image signal, and fordetecting a face area based on a matching degree with a contour plate ofa previously prepared face. In addition, one conventional patternrecognition technology is a method for detecting a face area byextracting feature points in a face, such as an eye, a nose, and amouth. The face recognition process may be executed by another methodthat is different from the above method.

The face detection processing circuit 209 sends a detection result (faceinformation) that contains object position information to the cameracontroller 207. The object position information is defined asinformation on the position of the object area in the captured image.The camera controller 207 sends information to the focus signalprocessor 204 based on the received detection result so as to add thearea used for the focus detection to the position that contains the facearea in the captured image.

The camera controller 207 sends information, such as the face areaposition and size, to the camera signal processor 203 so as to informthe photographer of the face area recognized by the face detectionprocessing circuit 209, and displays a face frame corresponding to thisinformation on the display unit 205 while the face frame is superimposedon the image signal.

The camera operating unit 208 serves to switch the control (facedetection control) by the face detection process between the valid stateand the invalid state. When the face detection control is valid, thecamera controller 207 performs the control in accordance with the facedetection process result about the focus assisting control etc. Inaddition, the camera operating unit 208 serves to select and specify theobject in accordance with the operating input of the photographer whowishes to fix an object to a specific object. When the photographerselects and specifies a specific face using an operating member, such asa cross key and a touch panel, the camera controller 207 performs afocus assisting control etc. for the selected and specified face(referred to as a “main face selection” hereinafter).

(Structure of Image Sensor)

FIG. 2 illustrates a 4 column×4 row range of a pixel array in atwo-dimensional CMOS sensor used for the image sensor 201 (where thefocus detecting pixel array has an 8 column×4 row range).

In this embodiment, the pixel unit 200 includes a 2 column×2 row pixelarray covered by Bayer-arrayed color filters. In the pixel group 200, apixel 200R having a R (red) spectral sensitivity is disposed at an upperleft position. A pixel 200G having a G (green) spectral sensitivity isdisposed at upper right and lower left positions. A pixel 200B having aB (blue) spectral sensitivity is disposed at a lower right position. Theimage sensor 201 provides an imaging plane phase difference type offocus detection, and each pixel includes a plurality of photodiodes(photoelectric converters) for one micro lens 215. In this embodiment,each pixel has two photodiodes 211 and 212 arranged in a 2 row×1 columnmatrix.

The image sensor 201 can acquire the capture signal and the focusingsignal by arranging many pixel groups 200 each of which includes a 2column×2 row pixel array (or a 4 column×2 row photodiode array).

Each pixel in this embodiment separates a light flux through the microlens 215 and receives through the photodiodes 211 and 212. A signal madeby adding signals from the photodiodes 211 and 212 to each other (A+Bimage signal) is used for the capture signal, and the two signals (A andB image signals) read out of the photodiodes 211 and 212 are used forthe focusing signals. The capture signal and the focusing signal may beseparately read out, but the image signal (A+B image signal) and the oneof the signals (such as the A image signal) in the focusing signals fromamong the photodiodes 211 and 212 so as to lessen the processing load.The other signal (such as the B image signal) can be acquired bycalculating a difference between these signals.

Each pixel in this embodiment has the two photodiodes 211 and 212 forone micro lens 215, but the number of photodiodes is not limited to twoand may be more than two. A plurality of pixels in which light receivershave different opening positions may be used for the micro lens 215. Inother words, another structure may be employed as long as it providestwo signals, such as the A image signal and the B image signal, for thephase difference detection. In addition, every pixel in this embodimenthas a plurality of photodiodes as illustrated in FIG. 2, but the focusdetecting pixels may be discretely provided among normal pixels in theimage sensor 201.

First Embodiment

(Focus Assisting Display Modes)

Referring to FIGS. 3A to 3D, a focus assisting display mode according tothis embodiment will be described. In this embodiment, there are fourfocus assisting display types or the first to fourth display modes, anddisplay parts 301 to 317 express the detected focus state.

FIG. 3A illustrates one example of the first display mode in which it isdetermined that the object is focused, and the position of the inwarddisplay part 301 and the position of the outward display part 302 arealigned with each other at the top. In addition, for example, thedisplay part 301 and the display part 302 may be expressed in differentcolors, such as white and green.

FIG. 3B illustrates one example of the second display mode in which thefocus detecting result has high reliability although the object is notfocused, and FIG. 3B shows a direction to an in-focus position and adefocus amount. For example, when the infinity side of the object isfocused (back focus), the inward display part 303 stops at the top andthe outward display parts 304 and 305 move laterally symmetrically alongthe circumference. The positions of the display parts 304 and 305indicate the defocus amount, and as both are more distant from theposition of the display part 303 (reference position), the defocusamount is larger. The display part 303 corresponds to the display part301, and the display parts 304 and 305 which are superimposed on eachother correspond to the display part 302.

On the other hand, when the near side of the object is focused (frontfocus), the outward display part 306 stops at the top, and the inwarddisplay parts 307 and 308 laterally symmetrically move along thecircumference. The positions of the display parts 307 and 308 indicatethe defocus amount, and as both are more distant from the position ofthe display part 306 (reference position), the defocus amount is larger.The display part 306 corresponds to the display part 302, and thedisplay parts 307 and 308 which are superimposed on each othercorrespond to the display part 301.

As described above, the defocus amount can be expressed by the positionof the moving display part in the second display mode. In addition, thedirection towards the in-focus position (defocus direction) can beexpressed by the orientation of the display part that stops at the top.

FIG. 3C illustrates one example of the third display mode in which thefocus detecting result has intermediate reliability, and FIG. 3C showsthe direction to the in-focus position. In this state, irrespective ofthe defocus amount, the display parts 309 to 314 are fixed atpredetermined positions. In the back focus, the inward display part 309stops at the top, whereas in the front focus, the outward display part312 is fixed at the top. In other words, the third display mode does notindicate the magnitude of the defocus amount and illustrates thedirection to the in-focus position through the orientation of thedisplay part fixed at the top.

FIG. 3D illustrates one example of the fourth display mode in which thefocus detecting result has low reliability. FIG. 3D does not show thedefocus amount or defocus direction and makes the user visuallyrecognize that the focus detection fails. In this state, the displayparts 315 to 317 are expressed by different colors from those of theother examples, such as gray, and the fixed at the predeterminedpositions. In addition, the display parts 316 and 317 have shapesdifferent from those of the other display modes.

FIGS. 3A to 3D merely illustrate examples of the focus assistingdisplays, and the present invention is not limited to these examples.

(Focus Assisting Display Control)

Referring now to FIG. 4, a description will be given of a focusassisting display control method executed by the camera controller 207.FIG. 4 is a flowchart of the focus assisting display control method. Thefocus assisting display control method according to this embodiment isexecuted every predetermined period in accordance with a computerprogram that runs on software and hardware. For example, it is executedevery readout period (every vertical synchronizing period) of thecapture signal from the image sensor 201 for generating the image forone frame (or one field). It may be repeated a plurality of times in thevertical synchronizing period. The computer program may be stored, forexample, in the camera controller 207 or a computer-readable storagemedium. While the camera controller 207 according to this embodimentexecutes the focus assisting display control method but a personalcomputer (PC) or a dedicated machine may execute, as the display controlapparatus, the focus assisting display control method according to thisembodiment. In addition, a circuit corresponding to the computer programaccording to this embodiment may be provided, and the focus assistingdisplay control method according to this embodiment may be executed byoperating the circuit.

In the step S101, the camera controller 207 acquires face informationfrom the face detection processing circuit 209.

In the step S102, the camera controller 207 sets a focus detecting areabased on the face information acquired in the step S101.

In the step S103, the camera controller 207 determines whether the focussignal has been updated in the focus signal processor 204. When it hasbeen updated, the flow moves to the step S104, and when it is notupdated, the flow moves to the step S115.

In the step S104, the camera controller 207 instructs the focus signalprocessor 204 to execute the focus detecting process and acquires thedefocus amount and the reliability as a result of the focus detectingprocess result.

A description will now be given of the focus detecting process executedby the focus signal processor 204. FIG. 5 illustrates a flowchart of thefocus detecting process according to this embodiment.

In the step S201, the focus signal processor 204 acquires a pair offocusing signals from the focus detecting area set in the step S102.

In the step S202, the focus signal processor 204 calculates acorrelation amount based on a pair of focusing signals acquired in thestep S201.

In the step S203, the focus signal processor 204 calculates a focuschange amount based on the correlation amount calculated in the stepS202.

In the step S204, the focus signal processor 204 calculates a focusshift amount based on the correlation change amount calculated in thestep S203.

In the step S205, the focus signal processor 204 calculates thereliability of the focusing signal acquired in the step S201. Theliability represents how reliable the focus shift amount calculated inthe step S204 is.

In the step S206, the focus signal processor 204 converts the focusshift amount into a defocus amount.

The defocus amount may be expressed by an absolute distance from thein-focus position, or the number of necessary pulses for moving thefocus lens 103 to the in-focus position, or an index having a differentdimension and unit, or a relative index. In other words, the defocusamount may represent a determination criterion of the separation degreefrom the in-focus state or a focus control amount necessary for thein-focus state.

Referring now to FIGS. 6A to 8, a detailed description will be given ofthe focus detecting process. FIG. 6A illustrates one example of thefocus detecting area 402 set on a pixel array 401 in the image sensor201. A calculation area 404 used to read the focusing signal necessaryfor the following correlation calculation includes a combination of afocus detecting area 402 and shift areas 403 necessary for thecorrelation calculation. In FIG. 6A, each of p, q, r, and t represents acoordinate in the x-axis direction, and the calculation area 404 rangesfrom p to q, and the focus detecting area 402 ranges from s to t.

FIGS. 6B to 6D illustrate one example of the focusing signal acquiredfrom the calculation area 404 in FIG. 6A. In each figure, a range from sto t corresponds to the focus detecting area 402, and a range from p toq corresponds to the calculation area 404 based on the shift amount. Asolid line 501 represents the A image signal, and a broken line 502represents the B image signal.

FIG. 6B illustrates waveforms of the A image signal 501 and the B imagesignal 502 before they are shifted. FIG. 6C illustrates the waveforms ofthe A image signal 501 and the B image signal 502 shifted in thepositive direction from those illustrated in FIG. 6B. FIG. 6Dillustrates the waveforms of the A image signal 501 and the B imagesignal 502 shifted in the negative direction from those illustrated inFIG. 6B. In calculating the correlation amount, each of the A imagesignal 501 and the B image signal 502 is shifted one bit at a time inthe arrow direction.

Next follows a description of a correlation amount COR calculatingmethod in the step S202 in FIG. 5. The focus signal processor 204 shiftsthe A image signal 501 and the B image signal 502 one bit at a time, andcalculates an absolute value of a sum of a difference between the Aimage signal 501 and the B image signal 502 in the set focus detectingarea 402 in each shift state. Herein, a minimum shift number is p-s, anda maximum shift amount is q-t. Where “i” is a shift amount, “x” is astarting coordinate of the focus detecting area, and “y” is an endingcoordinate of the focus detecting area, the correlation amount COR iscalculated as follows:COR[i]=Σ_(k=x) ^(y) |A[k+i]−B[k−i]|{(p−s)<i<(q−t)}  (1)

FIG. 7A illustrates one example of a change of the correlation amount,where the abscissa axis in the graph represents the shift amount and theordinate axis represents the correlation amount. In a correlation amountwaveform 601, reference numerals 602 and 603 denote segments around peakvalues. As the correlation amount is smaller, the coincidence degreebetween the A image signal 501 and the B image signal 502 becomeshigher.

Next follows a description of a correlation change amount ΔCORcalculating method in the step S203 in FIG. 5. The focus signalprocessor 204 calculates the correlation change amount based on thecorrelation amount difference of one shift skip and on the correlationamount waveform 601 in FIG. 7A. In this case, the minimum shift numberis p−s in FIGS. 7A and 7B, and the maximum shift number is q-t in FIGS.7A and 7B. Where “i” represents the shift amount, the correlation changeamount ΔCOR is calculated as follows:ΔCOR[i]=ΔCOR[i−1]−ΔCOR[i+1](p−s+1)<i<(q−t−1)  (2)

FIG. 7B illustrates one example of the correlation change amount ΔCOR,where the abscissa axis in the graph represents the shift amount and theordinate axis represents the correlation amount. In a correlation amountwaveform 604, reference numerals 605 and 606 denote segments containinginflection points of the correlation change amount from plus to minus.In the segments 605 and 606, the state in which the correlation changeamount is zero is referred to as a zero cross. In this state, thecoincidence degree between the A image signal 501 and the B image signal502 is highest and the focus shift amount is calculated based on theshift amount.

FIG. 8A is an enlarged view of the segment 605 in FIG. 7B, and referencenumeral 607 denotes part of the correlation change amount waveform 604.Referring now to FIG. 8A, a description will be given of a focus shiftamount PRD calculating method in the step S204.

The focus shift amount PRD includes an integer part β and a decimal partα. The decimal part α is calculated as follows based on a similarityrelationship between a triangle ABC and a triangle ADE in FIG. 8A.

$\begin{matrix}{{{{AB}\text{:}{AD}} = {{BC}\text{:}{DE}}}{{{\Delta\;{{COR}\left\lbrack {k - 1} \right\rbrack}\text{:}\Delta\;{{COR}\left\lbrack {k - 1} \right\rbrack}} - {\Delta\;{{COR}\lbrack k\rbrack}}} = {{\alpha\text{:}k} - \left( {k - 1} \right)}}{\alpha = \frac{\Delta\;{{COR}\left\lbrack {k - 1} \right\rbrack}}{{\Delta\;{{COR}\left\lbrack {k - 1} \right\rbrack}} - {\Delta\;{{COR}\lbrack k\rbrack}}}}} & (3)\end{matrix}$

The integer part β is calculated as follows based on FIG. 8A.β=k−1  (4)

The focus shift amount PRD is calculated based on the decimal part α andthe integer part β calculated in this way.

When there are a plurality of zero-crossing points as illustrated inFIG. 7B, a first zero cross is defined as a part having large steepnessmaxder of a correlation amount change at the zero cross (referred to as“steepness” hereinafter). The steepness is an index representing theeasiness of specifying the in-focus position, and a larger value meansthat the in-focus position is more likely to be specified. The steepnessis calculated as follows:maxder=|ΔCOR[k−1]|+|ΔCOR[k]|  (5)

As described above, when there are a plurality of zero crosses, thefirst zero cross is determined based on the steepness at the zero cross.

Next follows a description of a reliability calculating method of thefocusing signal in the step S205 in FIG. 5. The reliability correspondsto the reliability of the defocus amount. The following calculationmethod is one example, and the reliability may be calculated by anothermethod. The reliability can be defined as the above steepness and acoincidence degree fnc[v] between the A image signal and the B imagesignal (referred to as a “two-image coincidence degree” hereinafter).The two-image coincidence degree is an index representing the accuracyof the focus shift amount, and a smaller value means a higher accuracy.

FIG. 8B is an enlarged view of the part 602 in FIG. 7A, and thereference numeral 608 denotes part of the correlation amount waveform601. The two-image coincidence degree is calculated as follows:(i) Where ΔCOR[k−1]×2≤maxder,fnc[v]=COR[k−1]+ΔCOR[k−1]/4(ii) Where ΔCOR[k−1]×2>maxder,fnc[v]=COR[k]−ΔCOR[k]/4  (6)

In the step S104, when the focus detecting process ends, the flow movesto the step S105. In the step S105, the camera controller 207 determineswhether the defocus amount is smaller than a first predetermined amountand the reliability is higher than a first threshold Th_A. When thedefocus amount is smaller than the first predetermined amount and thereliability is higher than the first threshold Th_A, the flow moves tothe step S106 and when the defocus amount is larger than the firstpredetermined amount or the reliability is lower than the firstthreshold Th_A, the flow moves to the step S107.

The first predetermined value is used to determine whether the positionof the focus lens 103 is within an in-focus range for the object. In anexample, this embodiment sets the first predetermined value based on thedepth of focus. In addition, the first threshold Th_A is set to a levelsuch that the accuracy of the calculated defocus amount is reliable.When the reliability is higher than the first threshold Th_A, forexample, the A image signal and the B image signal have high contrastsand similar shapes (or the two-image coincidence degree is high) or themain object image is focused.

In the step S106, the camera controller 207 sets the focus assistingdisplay to the first display mode in FIG. 3A.

In the step S107, the camera controller 207 determines whether thedefocus amount is smaller than a second predetermined amount smallerthan the first predetermined amount and the reliability is higher thanthe first threshold Th_A. When the defocus amount is smaller than thesecond predetermined amount and the reliability is higher than the firstthreshold Th_A, the flow moves to the step S108 and when the defocusamount is larger than the second predetermined amount or the reliabilityis lower than the first threshold Th_A, the flow moves to the step S111.

In the step S108, the camera controller 207 calculates the indexorientation based on the defocus direction so as to set the indexrepresenting the direction and amount to the in-focus through the focusassisting display.

In the step S109, the camera controller 207 calculates the position fordisplaying the display part that moves in the second display mode inFIG. 3B based on the defocus amount.

In the step S110, the camera controller 207 sets the focus assistingdisplay to the second display mode in FIG. 3B.

In the step S107, the second predetermined value is a defocus valuedetected irrespective of the object. For example, a detectable defocusamount is different between a high-contrast object and a low-contrastobject. In this case, the displayable state is different in the seconddisplay mode depending on the object, and the user may feel strange.Hence, the second predetermined amount is an amount that provides thedefocus amount to some extent irrespective of the object. Thisembodiment sets the defocus amount to 2 mm in an example. However, thesetting method is not limited and is different according to the shiftamount in calculating the focus shift amount. Where the defocus amountin which the shift amount exceeds 2 mm cannot be detected, the settingis unnecessary and the second predetermined amount may be extremelylarge.

When the defocus amount may be determined based on the operability ofthe focus assisting display. In the second display mode, the movingdisplay part represents a shift from the in-focus state. Hence, when thelarge shift from the display part fixed at the top is displayed, theuser is unlikely to recognize the distance to the in-focus position. Inaddition, when the focus assisting display size is larger due to thedisplay method, it becomes difficult to recognize the image and thus thedefocus amount may be determined based on these factors.

In the step S111, the camera controller 207 determines whether thereliability is equal to or lower than a second threshold Th_B. When thereliability is equal to or lower than the second threshold Th_B, theflow moves to the step S114, and when the reliability is higher than thesecond threshold Th_B, the flow moves to the step S112.

In the step S112, the camera controller 207 calculates the orientationof the index of the focus assisting display based on the defocusdirection.

In the step S113, the camera controller 207 sets the focus assistingdisplay to the third display mode in FIG. 3C.

Thus, when the reliability is lower than the first threshold Th_A andhigher than the second threshold Th_B or when the reliability isintermediate, it is determined that the defocus direction representingthe direction in which the in-focus position is likely to exist isaccurate. With the intermediate reliability, the two-image coincidencedegree calculated by the focus signal processor 204 is lower than thepredetermined value but the correlation amount obtained by shifting theA image signal and the B image signal to each other has a certain trendand the defocus direction is reliable. For example, a small bluer occursin the main object.

In the step S114, the camera controller 207 sets the focus assistingdisplay to the fourth display mode in FIG. 3D since both the defocusamount and the defocus direction are unreliable. With the reliabilityequal to or lower than the second threshold Th_B, for example, the Aimage signal and the B image signal have low contrasts and the two-imagecoincidence degree is low. In this state, the object is significantlydefocused and it is difficult to calculate the defocus amount.

In the step S115, the camera controller 207 sets the parameter necessaryfor the focus assisting display, such as color information in the focusassisting display and the orientation and position of the index, basedon the display mode set by the above process, and notifies the displayunit 205 of it.

Referring now to FIG. 9, a description will be given of one example of adisplay position calculating method for the display part in the focusassisting display in the step S109 in FIG. 4. FIG. 9 is a relationalview between the defocus amount and the display part position (indexposition) for the focus assisting display. In FIG. 9, the abscissa axisrepresents the defocus amount, and the ordinate axis represents theindex position. The index position is a moving amount (angle) of thedisplay part (304, 305, 307, and 308) that moves relative to thereference display part (303, 306) fixed at the top according to thesecond display mode in FIG. 3B.

Assume the constant detection accuracy irrespective of the detecteddefocus amount. Then, where a relationship between the defocus amountand the index position is linearly expressed as illustrated by a dottedline like 702 in FIG. 9, the focusing operation becomes easy since theoperability accords with the display position of the display part.However, in the imaging plane phase difference detection method, as thedefocus amount is larger, the focus detection accuracy becomes lower.Hence, when the display part is displaced at the position linearlycorresponding to the detected defocus amount, the relationship betweenthe actual focus state and the display part position may conceivablyscatter as the detection accuracy becomes lower. In this case, the usermay feel unpleasant and the operability may lower.

Hence, as the defocus amount is larger as illustrated by a solid line701 in FIG. 9, the display part position becomes less likely to move. Inother words, as the defocus amount is larger, the display part positionchange (conversion amount) corresponding to the defocus amount is madesmaller. In other words, as the defocus amount is larger, the defocusamount corresponding to the position change per the display part unitangle (unit moving amount) is made larger.

For example, in the focus assisting display expressed by the solid line701, when the display part position is expressed by an angle, thedefocus amount of 0.02 mm per one degree is used to express the defocusamount up to 0.5 mm. The defocus amount of 0.04 mm per one degree isused to express the defocus amount up to 1 mm, and the defocus amount of0.08 mm per one degree is used to express the defocus amount up to 2 mm.

Assume that one degree is expressed based on the depth of focus. Then,one degree expresses the depth of focus until the defocus amount becomes0.5 mm. One degree expresses a double value of the depth of focus untilthe defocus amount becomes 1 mm. One degree expresses a quadruple valueof the depth of focus until the defocus amount becomes 2 mm.

Thus, the index position control can realize a stable focus assistingdisplay irrespective of the defocus amount. The index position controlis not limited to this example, and is variable according to adiaphragm, an imaging scene, etc.

Referring now to FIG. 10, a description will be given of the faceinformation acquiring process in the step S101 in FIG. 4. FIG. 10 is aflowchart illustrating a procedure of selecting a main face from among aplurality of faces detected by the face detection processing circuit 209and for outputting the information of the selected main face positionand size.

In the step S301, the camera controller 207 determines whether the facedetection processing circuit 209 has detected the face. When the facehas been detected, the flow moves to the step S304, and when the facehas not yet been detected, the flow moves to the step S302.

In the step S302, the camera controller 207 acquires the focus assistingdisplay position information displayed on the current image.

In the step S303, the camera controller 207 turns off the main facefixing mode.

In the step S304, the camera controller 207 determines whether the mainface fixing mode is turned on. When the main face fixing mode is turnedon, the flow moves to the step S309, and when the main face fixing modeis turned off, the flow moves to the step S305.

In the step S305, the camera controller 207 determines whether or notthe MF is being operated. When the MF is being operated, the flow movesto the step S306, and when the MF is not being operated, the flow movesto the step S309.

In the step S306, the camera controller 207 determines whether the mainface was set last time or in the one-frame previous process. When themain face was set, the flow moves to the step S307, and when the mainface was not set, the step moves to S308.

In the step S307, the camera controller 207 turns on the main facefixing mode. Turning on the main face fixing mode prohibits the mainface from being switched to another face by turning on the main facefixing mode.

In the step S308, the camera controller 207 acquires the positioninformation of the focus assist display displayed on the current imagesimilar to the step S302.

In the step S309, the camera controller 207 executes the main facedetermining process.

In the step S310, the camera controller 207 acquires the position andthe size information of the face that is determined to be the main face.

Referring now to FIG. 11, a description will be given of the main facedetermining process in the step S309 in FIG. 10. FIG. 11 is a flowchartof the main face determining process. In the main face determiningprocess, the camera controller 207 and the face detecting processingcircuit 209 serves as the main object determining unit for determiningthe main object among objects contained in the captured image based onthe image signal output from the imaging part (image sensor 201).

In the step S401, the camera controller 207 performs a face correlationdetermining process based on the face detecting process result by theface detection processing circuit 209. More specifically, the cameracontroller 207 assigns the face number to each face detected by the facedetecting process.

In the step S402, the camera controller 207 sets the face that has thehighest priority to the main face through the priority order of the facein accordance with the detected face position and size. The cameracontroller 207 determines the priority order of the face, for example,so that the larger face closer to the image center gets a higherpriority. Alternatively, the camera controller 207 may make higher thepriority order of the face also detected last time. Thereby, thefrequent switching of the detected face can be restrained. The cameracontroller 207 can use an arbitrary priority ordering method as long asthe photographer feels that the set main face is suitable for the mainface by the method.

In the step S403, the camera controller 207 determines whether the mainface fixing mode is turned on. When the main face fixing mode is turnedon, the flow moves to the step S404, and when the main face fixing modeis turned off, the process ends by setting the first priority face tothe main face.

In the step S404, the camera controller 207 compares the face numberassigned to the face obtained by the current face detection processingresult with the registered main face fixing face numbers, and determineswhether there is a face fixedly set as the main face. When the facenumber assigned to the face obtained by the current face detectionprocessing result has the face number that accords with the registeredmain face fixing face number, the camera controller 207 determines thatthere is a face fixedly set as the main face and moves to the step S405.When the face number assigned to the face obtained by the current facedetection processing result does not have a face number that accordswith the registered main face fixing face number, the camera controller207 determines that the face fixedly set as the main face hasdisappeared and moves to the step S406.

In the step S405, the camera controller 207 sets the face correspondingto the registered main face fixed face number to the first priority face(main face).

In the step S406, the camera controller 207 clears the facecorresponding to the registered main face fixing face number.

In the step S407, the camera controller 207 turns off the main facefixing mode. When the main face fixing mode is turned off, switching ofthe main face to another object face is permitted.

Referring now to FIG. 12, a description will be given of the facecorrelation determining process in the step S401 in FIG. 11. FIG. 12 isa flowchart of the face correlation determining process. The cameracontroller 207 repetitively executes the following face correlationdetermining process a plurality of times corresponding to the number ofdetected faces.

In the step S501, the camera controller 207 compares the previous facedetection processing result with the current face detection processingresult.

In the step S502, the camera controller 207 determines whether thecurrently detected face is the same as the previously detected face.When the currently detected face is the same as the previously detectedface, the flow moves to the step S503, and when the currently detectedface is different from the previously detected face, the flow moves tothe step S504. The camera controller 207 provides the comparison basedon the detected face position and size, and determines that they are thesame faces as the current face position is closer to the previous faceposition and as the current face is larger than the previous face. Morespecifically, the camera controller 207 calculates the correlationamount used to determine whether they are the same faces based on thepositional difference and the size difference between the faces, anddetermines that they are the same faces when the correlation amount ishigh.

In the step S503, the camera controller 207 sets the same number as thatof the previously detected face to the currently detected face.

In the step S504, the camera controller 207 sets a new arbitrary facenumber to the currently detected face.

Thereby, the main face fixing mode is turned on when the face isdetected and the MF is operated, and the main face fixing mode ismaintained until the current main face is not detected. Hence, after theMF operation ends, the target object for the focus assisting displaycontrol is never prevented from being suddenly switched. In addition,where the main face does not exist (or is not detected) after the MFoperation starts, the focus assisting display is not controlled for themain face. Thus, the target object for the focus assisting display isnever switched in the MF operation, and the main face is stably focused.

FIG. 13 illustrates a variation of the process in the face informationacquiring process in FIG. 10 from when the face is not detected to whenthe main face fixing mode is turned off. The process after the step S602(or the steps S620 to S624) is different from the face informationacquiring process in FIG. 10. The other processes are similar and thus adetailed description thereof will be omitted.

In the step S620, the camera controller 207 determines whether the MF isbeing operated. When the MF is being operated, the process ends, andwhen the MF is not being operated, the flow moves to the step S621.

In the step S621, the camera controller 207 determines whether the mainface fixing mode turns on and the post-MF operation counter is smallerthan the predetermined count value. When the main face fixing mode turnson and the post-MF operation counter is smaller than the predeterminedcount value, the flow moves to the step S622, and when the main facefixing mode turns off or when the post-MF operation counter is largerthan the predetermined count value (after the predetermined timeelapses), the flow moves to the step S623. When the post-MF operationcounter is equal to the predetermined count value, the destination stepcan be arbitrarily set.

In the step S622, the camera controller 207 counts up the post-MFcounter.

In the step S623, the camera controller 207 turns off the main facefixing mode.

In the step S624, the camera controller 207 clears the post-MF operationcounter.

Thereby, even when the current main face temporarily disappears, themain face fixing mode is turned off so as to prevent the main face fromswitching for manual focusing with the stable focus assisting display.

According to this embodiment, the imaging apparatus having the focusassisting function in the manual focusing can turn on the main facefixing mode when the manual focusing starts on a human face by using thehuman face detecting function. In addition, this embodiment does notturn off the main face fixing mode while the current main face existsand before a predetermined time elapses after the current main facedisappears. This configuration can prevent the focus assisting displayon an object different from the object targeted by the photographer.Thereby, the photographer is less likely to feel unpleasant because ofthe stable focus assisting display and the improved operability in themanual focusing.

Second Embodiment

FIG. 14 is a flowchart of a face information acquiring process accordingto this embodiment. The face information acquiring process according tothis embodiment is a variation of that in FIG. 10, and the facedetection processing circuit 209 serves as a specific object recognizerhaving a face recognition processing function that previously registersspecific human faces and identifies a registered face among the humanfaces. A description will now be given of differences, and a descriptionof a structure and control similar to those of the first embodiment willbe omitted.

In the step S701, the camera controller 207 determines whether the facehas been detected. When the face has been detected the flow moves to thestep S705, and when the face has not yet been detected, the flow movesto the step S702.

In the step S702, the camera controller 207 acquires the focus assistingdisplay position information on the captured image.

In the step S703, the camera controller 207 turns off the main facefixing mode.

In the step S704, the camera controller 207 releases the registeredauthenticated face.

When the face has not detected, the main face fixing mode may beprevented from turning off and the authenticated face may be preventedfrom being released in the MF operation and before a predetermined timeelapses after the MF operation ends, similarly to the process describedwith reference to FIG. 12.

In the step S705, the camera controller 207 determines whether the mainface fixing mode turns on. When the main face fixing mode turns on, theflow moves to the step S711, and when the main face fixing mode turnsoff, the flow moves to the step S706.

In the step S706, the camera controller 207 determines whether the MF isbeing operated. When the MF is being operated, the flow moves to thestep S707, and when the MF is not being operated, the flow moves to thestep S714.

In the step S707, the camera controller 207 determines whether the mainface was set last time. When the main face was set last time, the flowmoves to the step S708, and when the main face was not set last time,the flow moves to the step S709.

In the step S708, the camera controller 207 turns on the main facefixing mode.

In the step S709, the camera controller 207 acquires the focus assistingdisplay position information on the captured image.

In the step S710, the camera controller 207 registers the current mainface as the authenticated face.

In the step S711, the camera controller 207 performs a faceauthentication process. The face authentication process providesmatching between a currently detected face image and registered faceimages and outputs, as the authenticated face image, a face imagesimilar to the registered face image. The face authentication processmay use any conventional methods.

In the step S712, the camera controller 207 determines whether theauthenticated face exists. When the authenticated face exists, the flowmoves to the step S713, and when the authenticated face does not exist,the flow moves to the step S714.

In the step S713, the camera controller 207 sets the authenticated faceto the main face.

In the step S714, the camera controller 207 sets another result of themain face determining process to the main face.

In the step S715, the camera controller 207 acquires the current mainface position information.

According to this embodiment, the imaging apparatus having the focusassisting function in the manual focusing uses the face recognitionfunction that recognizes the registered specific human face incomparison with the first embodiment. Thereby, even when another objectcrosses in front of the main object, the main object can be againspecified without switching the main object. This embodiment can providea stable function, improve the operability, and prevent the photographerto feel unpleasant.

Advantages of the First and Second Embodiments

The first and second embodiments can prevent the main object targeted bythe photographer from being switched in manual focusing on the humanface by utilizing the face detection function and the focus assistingdisplay.

In addition, the first and second embodiments register the main objectby utilizing the specific face recognition function when the manualfocusing starts. Thereby, these embodiments can prevent the main objectfrom being switched to another object crossing in front of the mainobject.

This configuration can provide a display control apparatus that canprovide stable manual focusing by utilizing the focus assisting displaycontrol, and improve the operability of the photographer.

OTHER EMBODIMENTS

Embodiment(s) of the present invention can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2017-102497, filed on May 24, 2017, which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. A display control apparatus comprising: a memorydevice that stores a set of instructions; and at least one processorthat executes the set of instructions to function as: a focus detectorconfigured to detect a focus state based on an image signal acquiredfrom an imaging part; a main object detector configured to detect a mainobject among objects in an image based on the image signal output fromthe imaging part; a determiner configured to determine whether or notmanual focusing by a user is being performed; and a display controllerconfigured to display on a display unit an index representing the focusstate detected by the focus detector on the main object detected by themain object detector, wherein the display controller performs a controlso as to inhibit switching of the main object in a case where thedeterminer determines that the manual focusing by the user is beingperformed.
 2. The display control apparatus according to claim 1,wherein the display controller inhibits the main object from beingswitched when the main object detector detects the main object after themanual focusing starts.
 3. The display control apparatus according toclaim 1, wherein the display controller inhibits the main object frombeing switched while the main object detector detects the main objectafter the manual focusing ends.
 4. The display control apparatusaccording to claim 1, wherein the display controller allows the mainobject to switch before a predetermined time elapses after the manualfocusing ends and the main object detector does not detect the mainobject.
 5. The display control apparatus according to claim 1, whereinthe at least one processor further functions as a specific objectrecognizer configured to recognize a specific object in the image,wherein the main object detector detects as a current object an objectrecognized by the specific object recognizer, and wherein the specificobject recognizer registers the current main object as the specificobject when the manual focusing starts.
 6. The display control apparatusaccording to claim 5, wherein the specific object recognizer inhibits aregistration of the current main object from being released when themain object detector detects the current main object after the manualfocusing ends.
 7. The display control apparatus according to claim 5,wherein the specific object recognizer releases a registration of thecurrent main object a predetermined time after the main object detectordoes not detect the current main object after the manual focusing ends.8. The display control apparatus according to claim 5, wherein thespecific object recognizer releases a registration of the current mainobject until the manual focusing ends, if the main object detector doesnot detect the main object when the manual focusing starts.
 9. Thedisplay control apparatus according to claim 1, wherein the index isbased on a detection result of the focus state using the image signalcorresponding to the main object by the focus detector.
 10. An imagingapparatus comprising: an imaging part that includes a plurality ofpixels each having a plurality of photoelectric converters for one microlens, and is configured to receive a light flux from an imaging opticalsystem through the plurality of photoelectric converts and to output apair of image signals; a memory device that stores a set ofinstructions; and at least one processor that executes the set ofinstructions to function as: a focus detector configured to detect afocus state based on the pair of image signals through a phasedifference type of a focus detection; a determiner configured todetermine whether or not manual focusing by a user is being performed; amain object detector configured to detect a main object among objects inan image based on the image signal output from the imaging part; and adisplay controller configured to display on a display unit an indexrepresenting a focus state detected by the focus detector on the mainobject detected by the main object detector, wherein the displaycontroller performs a control so as to inhibit switching of the mainobject in a case where the determiner determines that the manualfocusing by the user is being performed.
 11. A display control methodcomprising: a focus detection step of detecting a focus state based onan image signal acquired from an imaging part; a main object detectionstep of detecting a main object among objects in an image based on theimage signal output from the imaging part; a determination step ofdetermining whether or not manual focusing by a user is being performed;and a display control step of performing control so as to display on adisplay unit an index representing the focus state detected by the focusdetection step on the main object detected by the main object detectionstep, wherein in the display control step, switching of the main objectis inhibited in a case where the determination step determines that themanual focusing by the user is being performed.